Measurement invariance versus selection invariance : Is fair selection possible ? Denny Borsboom

This paper shows that measurement invariance (defined in terms of an invariant measurement model in different groups) is generally inconsistent with selection invariance (defined in terms of equal sensitivity and specificity across groups). In particular, when a unidimensional measurement instrument is used, and group differences are present in the location but not in the variance of the latent distribution, sensitivity and positive predictive value will be higher in the group located at the higher end of the latent dimension, whereas specificity and negative predictive value will be higher in the group located at the lower end of the latent dimension. When latent variances are unequal, the differences in these quantities depend on the size of group differences in variances, relative to the size of group differences in means. The effect is shown to originate as a special case of Simpson’s paradox, which arises because the observed score distribution is collapsed into an accept/reject dichotomy. Simulations show that the effect can be substantial in realistic situations. It is suggested that the effect may be partly responsible for overprediction in minority groups as typically found in empirical studies on differential academic performance. A methodological solution to the problem is suggested, and social policy implications are discussed.